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Universal Coding for Lossless and Lossy 
Complementary Delivery Problems 

Shigeaki Kuzuoka, Member, IEEE, Akisato Kimura, Senior Member, IEEE, 
and Tomohiko Uyematsu, Senior Member, IEEE 

Abstract 

This paper deals with a coding problem called complementary delivery, where messages from two correlated 
sources are jointly encoded and each decoder reproduces one of two messages using the other message as the side 
information. Both lossless and lossy universal complementary delivery coding schemes are investigated. In the lossless 
case, it is demonstrated that a universal complementary delivery code can be constructed by only combining two 
Slepian-Wolf codes. Especially, it is shown that a universal lossless complementary delivery code, for which error 
probability is exponentially tight, can be constructed from two linear Slepian-Wolf codes. In the lossy case, a universal 
complementary delivery coding scheme based on Wyner-Ziv codes is proposed. While the proposed scheme cannot 
attain the optimal rate-distortion trade-off in general, the rate-loss is upper bounded by a universal constant under some 
mild conditions. The proposed schemes allows us to apply any Slepian-Wolf and Wyner-Ziv codes to complementary 
delivery coding. 

Index Terms 

complementary delivery, multiterminal source coding, network coding, universal coding, Slepian-Wolf coding, 
Wyner-Ziv coding. 

I. Introduction 

The source coding problem for correlated information sources was initiated by Slepian and Wolf [1]. They treated 
the case where two information sources are encoded separately and then reproduced at the single destination. 
Subsequently, various coding problems derived from Slepian-Wolf coding have been considered (e.g. [2]-[5]). 
CoiTesponding lossy coding problem was studied by Wyner and Ziv [6], where they investigated the lossy coding 
problem when the decoder can fully observe the side information. While the messages are encoded separately in 
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Fig. 1. Complementary delivery problem 



Slepian-Wolf and Wyner-Ziv coding problems, the coding problems involving joint encoding processes has been 
also explored (e.g. [7]-[10]). 

This paper deals with a specific coding problem involving joint encoding, which is called complementary delivery 
coding. The block diagram of the complementary deUvery coding is depicted in Fig. [T] The encoder observes 
messages emitted from two correlated sources, and delivers these messages to two destinations (decoder 1 and 2). 
Each decoder reproduces one of two messages using the other message as the side information. Both lossless and 
lossy configurations have been considered. The lossless complementary delivery coding can be regarded as a special 
case of the coding problem investigated by Csiszar and Korner [11] and Wyner, Wolf and Willems [10]. Kimura 
et al. [12], [13] proposed a universal coding scheme for lossless complementary delivery based on graph coloring. 
The lossy complementary delivery problem was investigated by Kimura and Uyematsu [14], [15]. 

In this paper, we propose universal coding schemes for lossless and lossy complementary delivery coding 
problems. At first, we propose a simple construction of the lossless complementary delivery code based on Slepian- 
Wolf codes. The key idea of our coding scheme is as follows. We prepare two Slepian-Wolf codes. One of two 
codes is a Slepian-Wolf code for the source X with side information Y , which is used as the code from the encoder 
to the decoder 1 . The other code is a Slepian-Wolf code for the source Y with side information X, which is used as 
the code from the encoder to the decoder 2. Each source is encoded separately by the corresponding Slepian-Wolf 
code, and then, the encoder sends the summation of two codewords. Notice that, by using the side information Y, 
the decoder 1 can calculate the codeword from the encoder to decoder 2. Therefore, from the summation of two 
codewords, decoder 1 can extract the codeword to reproduce X. The decoder 2 can reproduce Y analogously. The 
above mentioned scheme allows us to apply any Slepian-Wolf code to lossless complementary delivery coding. 
This drastically enriches the variety of complementary dehvery coding. For example, we can use universal Slepian- 
Wolf codes (e.g. [16]-[18]). We can also apply Slepian-Wolf codes based on the low-density parity-check matrices 
(e.g. [19], [20]). In this paper, we demonstrate that a universal lossless complementary delivery code, for which the 
error probability is exponentially tight in some rate region, can be constructed by combining linear Slepian-Wolf 
codes [16]. 

Next, we propose a universal lossy complementary delivery coding scheme based on Wyner-Ziv codes [6]. Our 
scheme is universal in the sense that it does not depend on the joint probability distribution of the correlated sources. 
While our coding scheme cannot attain the optimal rate-distortion trade-off in general, the rate-loss is upper bounded 
by a universal constant under some mild conditions. Moreover, our scheme allows us to construct a universal lossy 
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Fig. 2. The source node observes the correlated sources {X, Y), and sends the message to the sink nodes 5 and 6 over the 
network. The node 3 (resp. 5, 6) corresponds to the encoder (resp. decoder 1,2) in Fig. [T] 

complementary delivery code by using (non-universal) Wyner-Ziv codes (e.g. [21]-[23]). 

The complementary delivery coding can be regarded as a special case of the network coding [24], [25]. Let us 
consider the network depicted in Fig. |2l The source node observes the messages emitted from the correlated 
sources {X,Y), and sends the message to the sink nodes 5 and 6 over the network. Assume that the all edges 
except the edge between the nodes 3 and 4 have sufficiently large capacity, and thus, the output from X (resp. Y) 
can be delivered to the nodes 3 and 6 (resp. 3 and 5). The problem is to find the minimum capacity between the 
nodes 3 and 4 satisfying that the codeword needed to reproduce X and Y can be delivered to the nodes 5 and 6. 
Then, this problem can be regarded as the complementary delivery problem depicted in Fig.[T] The node 3 (resp. 5, 
6) corresponds to the encoder (resp. decoder 1,2). The coding problem of correlated sources over a network was 
studied by Han [26]. In the recent years, considerable attentions have been devoted to Slepian-Wolf coding over a 
network (e.g. [27]-[31]). This paper shows that, for the specific network depicted in Fig. |2l the optimal code can 
be constructed by only combining two Slepian-Wolf codes. Further, the lossy complementary delivery investigated 
in this paper can be seen as a special case of lossy coding of correlated sources over a network, which is not so 
studied well as the lossless case. 

This paper is organized as follows. In Section |ll] we introduce definitions and notations used in this paper In 
Section Uni lossless complementary delivery coding is considered. We propose a simple construction of the lossless 
complementary delivery code based on Slepian-Wolf codes. Further, we propose another simple coding scheme 
which can work in a specific case. In Section |IV] lossy complementary delivery coding is considered. We propose 
a universal lossy complementary delivery coding scheme based on Wyner-Ziv codes. In Section [V] we present our 
conclusions and some directions for further work. 

II. Preliminary 

We denote by N a set of positive integers {1,2,...}. For a finite set S,\S\ denotes the cardinality of S. Throughout 
this paper, we take all log and exp to the base 2. We denote random variables by upper case letters such as X. 
Their sample values (resp. alphabets) are denoted by the corresponding lower case letters such as x (resp. calligraph 
letters such as X). For a random variable X, Px denotes the probability distribution of X. Similarly, for a pair 
of random variables {X,Y), the joint distribution is denoted by Pxy and the conditional distribution of Y give 
X is written by Py\x- For each n E N, X" denotes a random 7i-vector {Xi,X2, ■ ■ ■ , Xn), and = (xi, . . . , a;„) 
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denotes a specific sample value in Af" which is the n-th Cartesian product of X. A substring of a;" is written 
as xl = {xi, Xi+i, . . . , Xj) for i < j. When the dimension is clear from the context, vectors will be denoted by 
boldface letters such as a; e A"". 

A discrete memoryless source (DMS) is a sequence X = {Xi}"?!-^ of independent and identically distributed 
(i.i.d.) copies of a random variable X. For simplicity, we call a DMS X = {Xi}^!^ as a source X. In this paper, 
information theoretic quantities will be denoted following the usual conventions of the information theory literature 
(see, e.g. [32], [33]). The entropy rate of a source X is denoted by H{X). For a pair {X,Y) of correlated sources 
X and Y, the conditional entropy of Y given X is denoted by H{Y\X), and the mutual information between X 
and Y is denoted by I{X;Y). The relative entropy or divergence between two probability distributions P and Q 
is denoted by D{P\\Q). 

For a given pair (a;, y) & {X x y)'^ of sequences, the joint type of (a;, y) is defined as the empirical distribution 
Qxy of (x, y), that is, 

^ \{l<i<n:x, = a,y, = h}\ 
n 

for all (a, 6) e A* X 3^ [33]. Let Vn{X x be the set of all joint types of sequences in (A" x y^. By the type 
counting lemma [33, Lemma 1.2.2], we have 

\Vn{Xxy)\<{n+\)\''\\^\. (1) 

Hence, we can define an injection in'. Vn — > {1, 2, . . . , (n + l)!"^!!-^!}. (,„(P^^) is called the index assigned to 

PxY e Vnix x y). 

III. Lossless complementary delivery 

A. Previous results 

In this subsection, we formulate the lossless complementary delivery problem and show a fundamental bound of 
the coding rate. 

Definition 1: A lossless complementary delivery code of block length n is defined by a triple of mappings 

{fn,4>n\4>n^) where 

/„: X^^y-^Mn, 
<^«: A^„x3;"^A'", 
</.(f):7W„xAf"^3;", 

where Mn = {1,2, ... , ||/„||} and ||/„|| < oc. 

Definition 2: For a given pair [X, Y) of correlated sources X and Y, a rate R is said to be losslessly-achievable 
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if there exists a sequence {{fn,4'n\(t>r?)}^=i of codes satisfying 



limsup-log||/„|| < R, 

n — *oo 



limsupPr/x" ^ 0W (/„(X", F"), F")} = 0, 
limsupPrly" ^ ^/-i^) (/„(X",r"),X")| = 0. 



As a special case of results of [11] and [10], it can be shown that the infimum of the losslessly-achievable rate 
is given by max {H {X \Y), H{Y\X)}. Kimura et al. [12] proposed the universal coding scheme based on graph 
coloring which can achieve any rate R > max{H(X\Y), H{Y\X)}. 

Theorem 1 (Lossless coding theorem; direct part [12]): For a given rate _R, there exists a sequence {(/„, 
of a code such that for any {X, Y), 

limsup - log||/„|| < R 

n — *oo ^ 



and 



Pr 



(/„(x",r"),r")} 



< exp < —n 



min D{P^y\\PxY) ~ Q 



where 



and 



Sn{R) = i^PxY e VniX X y) : 

mSix{H{X\Y),H{Y\X)} > R 

C„ = -{|A'x3;|log(n + l) + l} 
n 

— » (n oo). 



On the other hand, the next theorem shows that the error exponent of the code appeared in Theorem [T] is tight. 

Theorem 1 
R, we have 



Theorem 2 (Lossless coding theorem; converse part [12]): For any code (/„, (t>n \ (j^n^) satisfying (l/n) log||/„ 



Pr 



(/„(x",r"),y")} 



+ Pr 
> exp •^ —n 



min D{Pj^y\\PxY) + Cn 



(2) 
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B. Universal coding based on Slepian-Wolf codes 

As shown in Section UlI-AI the coding scheme proposed in [12] is universal and optimal. However, it requires 
the exponentially large coding table. In this subsection, we propose a simple coding scheme based on Slepian-Wolf 
codes. 

At first, we consider Slepian-Wolf coding problem of a source X with side information Y. A Slepian-Wolf code 
of block length n for a source X with side information Y is defined by a pair of mappings {gn \ V'i^'') where 

and Mn = {1, 2, . . . , Ijiji^'' ||}. Similarly, we can define a Slepian-Wolf code {g'^\il>i?) for a source Y with 
side information X. The next lemma gives a simple construction of a lossless complementary delivery code from 
Slepian-Wolf codes. 

Lemma 1: For given Slepian-Wolf codes {g^n^^i^n^) and {gn\'^pn'^), there exists a lossless complementary 
dehvery code (/„, 4>n\4>n^) such that 

||/„||<max{||5W|U|5i^'l|} 

and 

Pr{x"/0« (/„(x",r"),y")} 

froq/:- Let M„ = max{||gi^-'||, and define /„, 4>n \ and ^i^^ by 

/„(a;,y)^gW(a;)©g(^)(y), 

where © (resp. 0) denotes the addition (resp. subtraction) in modulo A/„ arithmetic. The lemma follows from the 
construction of the code (/„, ip^n^)- □ 

Lemma [T] allows us to apply any Slepian-Wolf code to lossless complementary delivery problem. This drastically 
enriches the variety of complementary delivery coding. In the remaining part of this subsection, we demonstrate 
that a universal code which achieves the optimal rate \n.sy.{H{X\Y)^HiY\Xy} can be constructed by applying 
universal liner Slepian-Wolf codes [16] to Lemma [l] 
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Theorem 3: Assume that X = y and A" is a Galois field. Fix k (k < n) and let R = [k/n) log \X\. There exists 
a sequence {(/„, (t>n \ 0^)}5^Li of lossless complementary delivery codes such that for any (Jf, Y), 

1 



and 



-log||/„|| =i? 
n 



Pr{x"^0W (/„(x",y"),y")} 

< exp{-n(e;i(i?, Pxy) -£«)} 

< exp{-n(e^(i?, Pxy) -£«)} 



where 



and 



A 21og(n + 1) 
n 



\x\^\yY 



el{R,PxY) 

= min { D{P^y\\Pxy) + R-H{X\Y) 



el{R,PxY) 

A 



YaiulD{P^y\\PxY)+ R-H(Y\X 

PxY L } 

where the minimization is over all dummy random variables X and Y with joint distribution Pyy' 1^1^ ^ 
max(t, 0). 

Proof: Csiszar [16] showed that there exists a linear Slepian-Wolf code [gn\4'^^) for a source X with side 
information Y such that g'iP : A"" ^ A''^ and for any (X, F), 

Pr{x"^Vi') (5W(X"),r")} 

< exp {-n {el{R,PxY) - £«)} ■ 

Similarly, there exists a linear Slepian-Wolf code {g\ ) for a source y with side information X such that 

gi? ■■ y y'' and for any (X, Y), 

< exp{-n(e^(i?,Pxi') - £n)} ■ 

By applying two codes {gn\i^'ii^) and (.gi^\ V'n^'') to Lemma [H we have the theorem. □ 
Remark 1: As mentioned in Section |I] Theorem |3] can be seen as a result of Slepian-Wolf coding over a specific 
network (see Fig. |2|i. In [31, Theorem 6], Ho et al. also applied the linear coding approach in [16] to Slepian-Wolf 
coding over a network. However, there are some differences between our results and the result of [31]. While Ho 
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et al. considered more general networks than the network depicted in Fig. |2l Theorem 6 of [31] dealt with the 
special case where there exists only one receiver Hence, our result, Theorem [3] cannot be derived only by applying 
Theorem 6 of [31] to the network depicted in Fig. |2] Moreover, Lemma [T] allows us to apply not only liner but 
also various Slepian-Wolf codes to network coding. Further, the technique used in the proof of Lemma [T] can be 
applied to the lossy case which was not concerned in [31] (see Remark |4|l. 

Remark 2: In [12], the coding scheme based on graph coloring is applied to variable-rate coding for lossless 
complementary delivery problem. In the same way as the approach in [12], our coding scheme can be also modified 
and applied to variable-rate coding. For a given pair of sequences {x,y) to be encoded, let X and Y be random 
variables such that -Pj^y is identical to the join type of {x,y). Let R = max{H{X\Y), H{Y\X)} and (/„, 4'n \ (ji?) 
be a fixed-rate lossless complementary delivery code such that (1/n) log||/„|| = i?. If a; = <l)n\fn{x,y),y) and 
y = 4>n {fn{x,y),x), then, the encoder sends the codeword consisting of the flag bit "0", the index of 
PxY' ^^'^ fn{x, y). This codeword can be represented by using at most 1 + \Vn{<^ x y)| + -R bits. On the other 
hand, if a; ^ (j)n\fn{x,y),y) or y ^ {fn{x,y),x), then, the encoder sends the codeword consisting of the 
flag bit "1" and {x, y), which can be represented by using 1 + log \X x 3^|] bits. The overflow probability of the 
coding rate of this scheme can be bounded in the same way as an error probability of fixed-rate coding (see [12] for 
more details). Hence, it can be shown that, by using Slepian-Wolf codes, we can construct a universal variable-rate 
lossless complementary delivery code for which the coding rate is smaller than or equal to ma.x{H{X\Y), H{Y\X)} 
asymptotically almost surely. 

Now, we investigate the tightness of the error exponent of the proposed scheme. It is known that 

el{R,PxY)^ min D{Pxy\\Pxy) 

H{X\Y)>R 

el{R,PxY)= min D{Pxy\\Pxy) 

H{Y\X)>R 

if R < RIj. (i = 1, 2), where i?*,. = R].^.{Pxy) is the largest R for which the curve e\.{R, Pxy) meets its supporting 
line of slope one [16]. Hence, as a corollary of Theorem [3] we have 

Pr{x"^0W (/„(x",r"),y")} 

< 2exp|-nmini^(P^y||Pxy) -£«| (3) 

for R such that max{i/(X|F), H{Y\X)} < R < mini=i 2 Rl^- By comparing (|3]l with dU, it can be seen that the 
error bound (O is exponentially tight for R such that max{H{X\Y), H{Y\X)} < R < mini=i_2 i?cr- On the other 
hand, the error exponent bound for large rates can be improved in the same way as improving the error exponent 
of Slepian-Wolf coding. 

Theorem 4: Assume that X — y and X is a Galois field. Fix k (k < n) and let R — {k/n) log \ X\. There exists 
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a sequence {(/„, <j)n \ '/'n)}n^i of lossless complementary delivery codes such that for any [X, Y), 

1 



■log||/„ 



R 



and 



Pr{x"^0W (/„(x",r"),r")} 

< exp{-n(ei^(i?, Pxy) - £n)] 

< exp{-n(e^(i?,Fxy) -e™)} 



where 



el{R,PxY) 



A 



X:H{X)>R 



el{R,PxY) 



A 

= mm 
y://(y)>_R, 



X 



Ey 



logj^y PxY{x,y)PxY{xQX, y) 



-\og^ \J Pxy{x, y)PxY{x, yeY) 



+ R~ H{X) 



R - H{Y) 



where denotes the subtraction in the field X{= y) and E-^ (resp. Ey) denotes the expectation with respect to 
Px (resp. Py). 

Proof: By using Slepian-Wolf codes which attain the expurgated bound [16, Theorem 3], we can prove the 
theorem in the same way as Theorem |3] □ 

C. Binary symmetric case 

In this subsection, we propose another simple coding scheme for lossless complementary delivery problem 
which can work in a specific case. Let X = y ^ {0, 1}, and consider a binary symmetric source with parameter p 
(0 <p< 1/2), that is, 

i^, if a; = y, 
if x^y. 



PxY{xy) 



p 

2 ' 



In this case, a simple universal lossless code gives an optimal lossless complementary delivery scheme. For a given 
X e and y £ 3^", \et w ^ x Q) y, where © denotes the addition in modulo 2 arithmetic. Then, w can be 
regarded as an output from the source W = X (BY, which satisfies that Pw{0) — ^ ^ P ™d Pw{^) — P- It is well 
known that (see e.g. [33]) there exists a universal lossless code (/„, 0„) with rate R such that 



lim Pr{t^'V0n(/n(W^"))} =0 



10 



provided that R > h{p), where h is the binary entropy function defined as h{t) ~ —tlogt — {1 —t) log(l — t). By 
using {fm4'n), we can define the code {fn,<l> ' ) as 

fn{x,y) = fn{x®y), 
(j)''^\m,x) = (f>n{m) ®x. 

By the construction of the code, this simple code (/„, (t>n \ (j^i?) is universal. Further, it achieves the optimal rate 
h{p) = ma.x{H{X\Y), H{Y\X)} since h{p) = H{X\Y) = H{Y\X). Especially, if (/„,0„) is a lossless code for 
which the error exponent is tight (e.g. a code appeared in [33]), then, the error exponent of the code (/„, 4>n \ (t)n ^) 
based on (/„, 0„) is also tight, that is, (/„, (j)n \ (f^n^) attains the error exponent appeared in (|2]i. Furthermore, in 
the same way, we can also apply lossy codes to construct a lossy complementary delivery code (See the proof of 
Theorem [8] in Appendix IbTi. 

IV. Lossy complementary delivery 

A. Previous results 

In this subsection, we formulate the lossy complementary delivery problem and show a fundamental bound of the 
coding rate. Let X and y be reconstruction alphabets, and d^^^ : XxX ^ [0, rfmix] and S'^^ : yxy^ [0, dmL] be 
single-letter distortion functions (dmax < cxd (i = 1, 2)). Then, for each n £ N, the normalized distortion dn\x, x) 
between x G X" and x e <Y" is defined as 

n 

4}\x,x) = -yd(^\x,,xi). 

i=l 

^ (2) 

For y G 3^" and y G 3^", dn {y,y) is defined similarly. Now, we define codes for lossy complementary delivery 
problem. 

Definition 3: A lossy complementary delivery code of block length n is defined by a triple of mappings (/„ , (j)n'' , 4>n'' ) 
where 

/„: X-^xy^^Mu, 

(/-l^) : Mn X A-" ^ y\ 

where Mn = {1,2, ... , ||/„||} and ||/„|| < oo. 

Next, we define the achievability of rate and the optimal rate attained by lossy complementary delivery coding. 
Definition 4: For a given {X, Y) and a distortion pair (A^^^ A^^^), a rate R is said to be (A^^^, /S'^'^^)- achievable 
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if there exists a sequence {{fn,4'n\(t>r?)}^=i of codes satisfying 



limsup-log||/„|| < R, 

■n—>-oo ^ 



< A(2), 



n — *oo L \ / . 

\imsnpExY U'^ (y\4? (/„(X", F"), X"))" 

where Exy denotes the expectation with respect to Pxy- 

Definition 5: For a pair of sources {X,Y) and a pair of distortions (A^^^, A^^^), let 

i?*(X,r|A(i\ A(2)) ^ inf {i? : R is (A^^^ A(2))-achievable} . 

Kimura and Uyematsu [14], [15] revealed the optimal achievable rate for the lossy complementary delivery. 
Theorem 5 (Lossy coding theorem [14], [15]): For a given {X,Y), 

i?*(x,y|A(i), a(2)) 

= min [m£ix{I{X;U\Y),IiY;U\X)}] 

where the minimization is over all the auxiliary random variable U satisfying the following properties: 

1) PxYu{x,y,u) = PxY{x,y)Pu\xY{u\x,y), 

2) U takes a value over an alphabet U satisfying < jA:" x + 2, and 

3) there are functions (/s'^' : U x y ^ X and tp^'^'> : U x X ^ y satisfying 

A^''^>ExYU [S'Hx,ip^'\u,Y)) 
A'-^^>ExYU \d^^HY,^^^\u,X)) 



B. Universal coding based on Wyner-Ziv codes 

The coding scheme appeared in the direct part of the proof of Theorem |5] depends on the joint distribution 
Pxy of {X, Y). We propose a lossy complementary delivery coding scheme which does not depend on the joint 
distribution. 

At first, we consider Wyner-Ziv coding problem of a source X with side information Y under the distortion 
constraint A*^^) associated with the distortion measure d^^-* : X x X ^ [0, o?max]- A Wyner-Ziv code of block length 
n for a source X with side information Y is defined by a pair of mappings {gn\'ipn'') where 

.(1) 



and Mn = {1,2,..., 115^11}. Define Rwz{X,Y\d(^\ A^^^) by 



Rwz{X,Y\S^\A('')) = min {/(X; [/) - /(F; [/)} 

Pu\x 
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where the minimization is over all the random variables U satisfying the following properties: 

1) PxYu{x,y,u) = Puix{u,x)PxY{x,y), 

2) \U\ < \X\ + 1, and 

3) there exists a function ip: U x y ^ X satisfying that 

ExYu[d'^'Hx,ip{U,Y))] < A(i). (4) 

To simplify the notation, we denote Rwz{X,Y\d'^''-\ A^''-^) by r'^^{A'^'^\ Pxv)- It is known that the optimal 
coding rate which can be achieved by a Wyner-Ziv code for a source X with side information Y under the 
distortion constraint A^^^ is given by r\^^{A^^\ Pxy)- 

Theorem 6 ( [6], [33]): For any S > and any {X,Y), there exists = ^o(<5, ^max, \X\, \y\) such that for any 
I > Iq there exists a code (g^-',?/'/^'') satisfying 

ilog||5«||<i?«^(AW,P^^) + (5 

and 

Pr{dW (5«(X'),y'))>AW}<<5. 

Remark 3: While (g^^ depends on {X,Y), the virtue of the method of types [33] allows us to choose the 
block size I which depends only on (5,(imax,|'^|, and |3^|. 

In a similar manner to the above discussion, we can consider Wyner-Ziv coding problem of a source Y with side 
information X under the distortion constraint A*^^^ associated with the distortion measure 

max] ■ 

We denote RwziY, X\S^\ A^^'>) by i?^^^(A(2), P^y). 

Now, we describe a universal lossy complementary delivery coding scheme based on Wyner-Ziv codes. Fix 
7 > and R > 0. Choose (5 > such that S < and ASdmax < 7 (i = 1, 2). By Theorem |6] we can choose 
I = l{d, dmax, rfmax, \X\ , \y\) Sufficiently large so that, for any correlated sources {X, Y), there are (^P"*, V';^'') and 
V'f^) satisfying 

ilog||g«||<i?«^(A«,P_^^) + <5, ^ = l,2 (5) 

and 

Fi{X'y' ^ rz(P^y)} < 25 (6) 

where 

V,{P^y)'^^x'J): 

^ (x',V^W(5«(x'),y'))<AW, 
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Note that {gl"\i;l"'') (i = 1,2) may depend on Pxy- Especially, for each joint type Pxy^ '-^'^ choose the 
pair of codes {{gl^\ipi^'')}i=i.2 satisfying Q and (|6]l. For each n e N, fix a correspondence between VniX x y) 



and the set of pairs of Wyner-Ziv codes so that the pair {{gj^ , i/'j )}i=i,2 corresponding to G Vn{X x y) 

satisfies © and dSll Q. Let Af/ = 2'('"+3t/4). Let n be so large that n > ^ and \y\ /n) log(n + 1) < 7/4. In 
the followings, we assume that n — TI(Te N) for simpUcity. 

At first, we describe the encoding scheme. For a given (a;",?/"), find Pxy ^ ^^'-^ that[^ 

max (A(^) , P^^) < i? + 7/2 (7) 

and 

"<4(5r. (8) 



Note that is not necessarily the joint type of (a;",y"). If there is no P^y G P„ satisfying (|7]i and ([8]), 

then error is declared. If there exists Pxy ^ satisfying (|7]i and ([8]), then find the pair of Wyner-Ziv codes 
corresponding to Pj^y, that is, {((?;'''\ V'z*'')}i=i,2 satisfying Q and (|6]l. Parse (x", y") into P blocks of size I, and 
then, encode each block as 

where © denotes the addition in modulo M; arithmetic (Note that Mi > maxi=i_2||(?;^*''||). Then, the codeword 
assigned to (x", y") is (i(P^y), mo, . . . , tot-i)- Since ([T]i holds, the codeword can be described by using n(P+7) 
bits because 

log(n + l)!-^!!^! +TlogM; < n{R + j). 

Next, we describe the decoding scheme. We only describe the decoder ^1^' which outputs the reproduction 
sequence i" G A:"" by using the codeword (t(P^y), itiq, . . . , ttit-i) and the side information y". The decoder (/)„ 
can be defined analogously. (f)n ^ decodes the index (-{Pxy) ^^^^^ ^^'^ then, computes i" as 



,P-1 



where ip^'^ and gp-* are the mappings corresponding to Pj^y, and denotes the subtraction in modulo M; arithmetic. 
The next theorem shows the performance of the coding scheme described above. 

Theorem 7: Fix 7 > and P > 0. There exists a sequence {(/„, ^i^'' , ^i^' of lossy complementary delivery 
codes which satisfies the following property: If {X, Y) satisfies 

P> maxP«^(A«,Pxy) 

then, for sufficiently large n, 

-log||/„|| <P + 7 
n 



If there are two or more pairs of codes satisfying jsj and (6) for a joint type P^y' choose one of them arbitrarily and assign it to 
^If there are two or more joint types satisfying the conditions, choose one of them arbitraiily. 



P- - 
XY 
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and 

Pr{dW (/„(x",r"),y")) > aw +7} < 7, 

Pr (yn, 0(2) F"), X")) > A(2) + 7} < 7. 

Remark 4: The proposed scheme is universal in the sense that the scheme does not depend on the probability 
distribution Pxy of {X,Y). To deal with some technical difficulties in evaluating the performance of the code, 
we adopt the coding scheme which parses the sequence into blocks of fixed length I and then encodes each block. 
On the other hand, if we know the the joint distribution Pxy, we can avoid technical difficulties. In fact, a (non- 
universal) lossy complementary delivery code can be constructed by combining two Wyner-Ziv codes {gn'' , ipn"^ ) and 

f21 (2) 

(<?n , V'n ) in the same way as a lossless complementary delivery code is constructed by Slepian-Wolf codes (Lemma 
[1]). In the lossless case, we can use universal Slepian-Wolf codes to construct a universal lossless complementary 
delivery code. However, as long as the authors know, no universal Wyner-Ziv code has been proposed (While 
universal Wyner-Ziv coding was recently studied in [23], [34], it is assumed that the conditional distribution Py\x 
of the source is known). 

The proof of the theorem will be given in Appendix [A] 

Our coding scheme allows us to construct a universal lossy complementary delivery coding scheme based on 
(non-universal) Wyner-Ziv codes. Especially, we can apply practical Wyner-Ziv codes (e.g. [21]-[23]) to universal 
lossy complementary delivery. However, our scheme cannot attain the optimal rate appeared in Theorem|5]in general. 

Theorem 8: There exists {X, Y) such that 

maxi?[^^(A«,Pxy) > i?*(X,y|AW, A(2)). 

Remark 5: In the proof of Theorem [8] we give an example where a simple coding scheme appeared in Section 
Illl-Cl can attain the optimal rate F| A(^\ A^^^). See Appendix |B] for more details. 

On the other hand, we can show that the loss of the coding rate can be bounded by a universal constant under 
some conditions. Let ;f = i' = {1, 2, . . . , M^^)} and 3^ = j) = {1, 2, . . . , M^^'}. Suppose that (i = 1, 2) is a 
balanced distortion measure [35], that is, 

S''^a,a) = (i«(aea), Va,ae {1,...,mW} 

for some : {!,..., M^')} [0, diiL], where G denotes modulo-M^') subtraction. Let C^'^ (i = 1, 2) be the 
minimax capacity [36], defined as 

cW(aW)= inf sup I{W\W®N) 

Ar:_Ejv[!i(»)(Ar)]<A(») W:W^N, 

£w[<i<*'(lV)]<A(') 

where and W are random variables on {1, 2, . . . , Af'*'}, _L denotes statistical independence, and © denotes 
modulo- A/'*^ addition. The next theorem gives the bound on the rate-loss of our scheme. 
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Theorem 9: Suppose that X = X and 3^ = 3^. If and S-'^^ are balanced distortion measures, then 



max R^^z{^^'\Pxy) < R*{X, Y\A^^\A^^'>) + C'-''\A^'")) 



where i* = argmaxi=i,2 R'^^i'^^''' ^ ^xy)- 



Proofs of Theorem |8] and Theorem |9] will be given in Appendix |B] 



V. Conclusion 



We proposed a universal lossless (resp. lossy) complementary delivery coding scheme based on Slepian-Wolf 
(resp. Wyner-Ziv) codes. It was demonstrated that a universal lossless complementary delivery code, for which error 
probability is exponentially tight, can be constructed by only combining two linear Slepian-Wolf codes. On the 
other hand, proposed lossy complementary delivery coding scheme cannot attain the optimal rate generally, while 
it does not depend on the distribution of the source. The rate-loss of our lossy coding scheme was evaluated. We 
also propose another simple coding scheme which can work for a binary symmetric source. 

Further work includes extensions to generaUzed complementary delivery network [13]. Another important work 
is to construct a universal lossy complementary deUvery code which attains the optimal rate. 



A. Proof of Theorem [7| 

Before proving Theorem |7] we introduce some lemmas. In this appendix, the variational distance between Pxv 
and Pj^Y is denoted by p [Pxy , Pxy) ■ 

Lemma 2: For any 6 > and / E N, there exists ei = ei{l, 6, \X\ , \y\) > such that if 



Appendix 



Proofs of Theorems 



p{Pxy,Pxy) <ei 



then. 




Proof If p {Pxy,Pxy) < ei' then for any (x', y') e {X x y)'. 



Px'Y'{x',y'^) = '[[PxY{xt,yt) 



i=i 



<Y[{PxYi^^,y^)+^l} 



i=l 
I 



1=1 



Pj^,Y'i^^y) + ^(^i'i) 
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where S{ei,l) ^ as ei ^ 0. Similarly, we have P^'y' (2;', 2/0 > ^xiy; (a^', J/') ^ Hence, we have the 

lemma. □ 

Lemma 3: 1) For any 7 > and any Pxy, there exists C > satisfying 

i?«^(AW,P^^) < i?«^(AW +C,P;fy) + 7/4, * = 1,2. 

2) For any 7 > 0, C > and any (X, Y), there exists €2 > such that if p {PxY^ Pxy) ^ ^2 then 

R^^zi^^^ +CPxy) < <^z(A(^^^xy) +7/4, * = 1,2. 
Proof: The first part of the lemma follows from the fact that R^^(A^^\ ^xy) continuous in A^'^ [6]. 



By the definition of i?[^^^(A(i), Pxy), we can choose Pfyix and satisfying 



<)^(A(i) , Pxy) = I(X; U) - I{Y; U) 
and (|4|. If p{Pxy,Pxy) is sufficiently small, then P^y satisfies that 

/(X; [/) - /(f; [/) < /(X; U) - I{Y; U) + 7/4 



and 



Hence, we have 



p|^)^(AW +C,Pxy) < Pi^)^(AW, Pxy) +7/4. 



Similarly, we can prove that 

i?l^'^(AW +C,Pxy) < i?l^'z(A('^^xy) +7/4 
provided that p(Pxy,P^y) is sufficiently small. □ 

Proof of Theorem ^ We prove the theorem by showing that the code defined in Section IIV-BI satisfies the 
property appeared in the theorem. 

Let (5 > and ^ e N be numbers satisfying the conditions appeared in the description of coding scheme. Let 
n = e N be sufficiently large. 

Suppose that there exists P^y G Vn satisfying (O. Then, 

4'^ (x", (x",y"),y")) < i {4™dW, + r?A«} 

< a(i) +7. 

Similarly, 

4'^ (y",0(^)(/n(a;",y"),a;")) <A(2)+7. 
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Hence, to prove the theorem, it is sufficient to show that, for sufficiently large n, we can find Pjiy ^ satisfying 
dTji and (O with probabiUty greater than 1 — 7. 

Choose ei (resp. C,, £2) satisfying Lemma |2] (resp. Lemma |3]l. Fix e > such that e < (i = 1,2). If n is 
sufficiently large, there exists a joint type -P^y G Vn{<^ x satisfying 

By Lemma [3] we have 



Since R > niaxi=i.2 i?[^2(A(*),Pxy), P^y satisfies ©. In the foUowings, we prove that ^Iso satisfies (H) 
with probability greater than 1 — 7. By Lemma |2l and (|9|l, 

Px'Y'irh<Px^Y^i^h + S<iS (10) 

where r[' denotes the complement of r;(Pyy). Let Bt (< = 0, 1, . . . , T — 1) be random variables defined by 

I 0, otherwise. 

Since (fTol l implies Ex'y^ [Bi] ^ 3(5, by the law of large numbers, we have 



0. 



Hence, \f n — Tl is sufficiently large, then we can find ^ satisfying (|7]i and (O with probability greater 
than 1 — 7. □ 

B. Proofs of Theorem |S] and Theorem [9] 

Before proving Theorem |8] and Theorem|9] we introduce some notations. Let i?[^|^(A'^\ Pxy) be the optimal 
rate of the lossy source coding problem with side information at the encoder and the decoder [6], that is, 

' Zl^) \XY 



Pil(AW,Pxy)^ min /(X;Z«|r) 



where the minimization is with respect to all random variables Z^^^ such that Pxyz<i) (^i ^) — Pxy{x, y)Pzw \xy{A^^ v) 
is the probability distribution on X x y x X satisfying ^ ^ S'^^ [x, z)Pxyzw i^i Vi ^) ^ A^^^ Similarly, let 

41(A(^),Pxy)^ min I{Y; Z^'^\X). 



Note that 



maxpWjAW, Pxy) 



< min [max{I{X;U\Y),I{Y;U\X)}] 

Pu I X Y 

= p*(x,r|A(i\A(2)) (11) 
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where minp^|^^ is taken over U satisfying the properties appeared in Theorem |5] 

Proof of Theorem^ Let X ^ y = X = y = {{),!}, and consider a binary symmetric source with parameter 
p {Q < p < 1/2). Let d!^^^ and d*^^) be the Hamming distortion measure, that is S-^\x^x) = if a; = a: and 
i) = 1 otherwise. Let A^^) = A'^) = A (A < p). 
For a given x G A:"" and y G 3^", \eX w = x ® y, where ® denotes the addition in modulo 2 arithmetic. Then, 
w can be regarded as an output from the source W ^ X ®Y , which satisfies that Pw{^) = 1 ^ P and Pw(l) = p. 
It is known that [35] there exists a lossy code (g„, -(/jn) with rate h{p) — ft.(A) satisfying that 

lim Prj^i) {W^,M9n{W^)) > A)| = 0. 
Based on {gm'ipn), define the code {fn,(l)n\(j)n'^) as 

fnix,y) =gn{x®y), 
(t)iP{m,y) = i'n{m)®y, 
0^^)(m, x) = V'„(to) © X. 

Since 

dW(a;,</.W(TO,y)) = 4i)(ti;,V;„(ff„(t«))) 

and 

4^) (y, (/-i^) (m, a^)) = d^f) 
the code (/„, (pn ^) satisfies the distortion constraints. This fact indicates that 

h{p)^h{A) > i?*(X,y|A(i\A(2)). (12) 
Further, by the result of lossy source coding with side information at the encoder and decoder [6], we have 

4L(AW,Pxy)=Mp)-MA), ^ = 1,2. 

Since (fTTT l holds, we have 

h{p)~h{^) < i?*(X,ylA(i),A(2)). (13) 

By combining (fT2l i and (flJT l. we have 

h{p)~h{A) = i?*(X,y|A(i),A(2)). (14) 
On the other hand, by the result of Wyner-Ziv coding problem [6], we have for each i = 1, 2, 

i?l^)^(A«,Pxy) = inf {0[/i(p*/3) - h{m (15) 

where the infimum is with respect to all 9, (3 such that i) < < 1, Q < (3 < p, and A = 6'/? + (1 — 0)p. ( fT5] l and 
(Ell indicate that max,=i^2 R'^ zi^^'^ ^ Pxy) > R*{X,Y\A^^\ A'^^l) for all < p < 1/2. □ 
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Proof of Theorem |9} Since (fTTT l holds, we can bound the rate loss as 



maxi?«^(A«,Pxy) 

z— 1,2 



i?*(x,y|A(i), a(2)) 



<i?l;:i(A(^*),Pxy) 

<i?l^*l(A('*),P^y) 



It is known that the difference i?[^*^(A(*'), Pxy)-P[*Jj)^(A(*'), Pxy) is bounded by a universal constant C('*)(A(**)) 
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